Visual coding for sensitivities to light, color and spatial resolution in human visual system

ABSTRACT

A visual coding system is contemplated to facilitate encoding and decoding of visuals, representations, images, etc. utilized to facilitate immersive, augmented reality (AR), virtual reality (VR), light field or other types of video. The visual coding system may facilitate coding, scaling, segmenting, tiling, or otherwise processing portions of individual frames differently depending on light, color and/or spatial resolution sensitivities of the human visual system (HVS), such as to account for the HVS processing light/color information unequally/differently across its field of view.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 15/419,858, filed Jan. 30, 2017, which claims the benefit ofand priority to U.S. Provisional Patent Application Ser. No. 62/288,538,filed Jan. 29, 2016, the disclosures of which are incorporated in theirentireties by reference herein.

This application claims the benefit of and priority to U.S. ProvisionalPatent Application Ser. No. 62/534,982, filed Jul. 20, 2017, thedisclosure of which is incorporated in its entirety by reference herein.

TECHNICAL FIELD

The present invention relates to visual coding systems, such as but notnecessary limited to visual coding systems utilized to facilitateencoding and decoding of visuals, representations, images, etc. utilizedto facilitate immersive, augmented reality (AR), virtual reality (VR),light field or other types of visual media.

BACKGROUND

Visual coding may be characterized as a process whereby uncompressedvisual media representations are modified, compressed or otherwise codedfor transmission and/or rendering, such as to facilitate subsequentdisplay on a screen or other mechanism suitable to facilitateinterfacing the related visual content with a viewer. High EfficiencyVideo Coding (HEVC) and Advanced Video Coding (AVC) are examples ofcompression standards for video (one particular form of visual media)coding having capabilities to facilitate implementing progressive andother forms of scalable coding tools. These and/or other coding toolsmay be utilized to facilitate scaling an uncompressed image to acompressed image according to different quality parameters, which in thecase of video may include frame size (resolution/detail), frame rate(fluidity of motion) and color depth (quantization or smoothness ofshading). (The term frame refers to a single image amongst a series ofimages that contains either the entire area of the media visible to theviewer, or a sub-portion (usually rectangular in shape) of the viewablearea. Those skilled in the art typically refer to the frame sub-portionas a tile.) The amount of data required to transmit a video followingcoding is proportional to application of the quality parameters, e.g., avideo coded at a greater frame size, frame rate or color depth willrequire more data for transmission than the same video coded at a lesserframe size, frame rate or color depth. Coding tools like HEVC and AVCbalance the quality parameters versus the attendant amount of dataneeded for transmission and thereafter apply scaling of thecorresponding images, which at a frame level is done uniformly such thateach frame has a consistent and a constant coding throughout.

The present invention contemplates accounting for coding of a frame (orframes) in a field of view that fail to sufficiently meet the perceptioncapabilities of a human visual system (HVS). The HVS processeslight/color information unequally/differently across a field of view,e.g., a person viewing a frame fails to process light/color equallyacross an entirety of the frame due to the HVS being more sensitive tolight in peripheral vision and more sensitive to color information at acenter field of view. One non-limiting aspect of the present inventioncontemplates visual coding portions of individual frames differentlydepending on light, color and/or spatial resolution sensitivities of theHVS to account for the HVS processing light/color informationunequally/differently across its field of view.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the presentdisclosure will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a schematic illustration of an exemplary system for deliveringimmersive visual content for a viewer, according to an embodiment.

FIG. 2 illustrates an exemplary content layout associating contentresolution with a viewer's field of view, in accordance with the systemdepicted in FIG. 1.

FIG. 3 illustrates a visual display system in accordance with onenon-limiting aspect of the present invention.

FIG. 4 illustrates a flowchart for a method for visual coding forsensitivities to light, color and spatial resolution in the HVS inaccordance with one non-limiting aspect of the present invention.

FIGS. 5-6 illustrate segmenting image frames according to saccadicmovement in accordance with one non-limiting aspect of the presentinvention.

Unless otherwise indicated, the drawings provided herein are meant toillustrate features of embodiments of this disclosure. These featuresare believed to be applicable in a wide variety of systems including oneor more embodiments of this disclosure. As such, the drawings are notmeant to include all conventional features known by those of ordinaryskill in the art to be required for the practice of the embodimentsdisclosed herein.

DETAILED DESCRIPTION

In the following specification and claims, reference will be made to anumber of terms, which shall be defined to have the following meanings.

The singular forms “a,” “an,” and “the” include plural references unlessthe context clearly dictates otherwise.

“Optional” or “optionally” means that the subsequently described eventor circumstance may or may not occur, and that the description includesinstances where the event occurs and instances where it does not.

Approximating language, as used herein throughout the specification andclaims, may be applied to modify any quantitative representation thatcould permissibly vary without resulting in a change in the basicfunction to which it is related. Accordingly, a value modified by a termor terms, such as “about,” “approximately,” and “substantially,” are notto be limited to the precise value specified. In at least someinstances, the approximating language may correspond to the precision ofan instrument for measuring the value. Here and throughout thespecification and claims, range limitations may be combined and/orinterchanged; such ranges are identified and include all the sub-rangescontained therein unless context or language indicates otherwise.

Immersive video, or virtual reality video, is visual content thatincludes some or all a panorama, and when viewed through a head-mounteddisplay or within surrounding screens, includes some or all a 360-degreepanorama in the horizontal and/or vertical directions. Some conventionalimmersive video systems include an immersive screen and a video storagesource. Immersive video delivered to the immersive screen from the videostorage source that is in the same location as the immersive screen, orfrom a remote location that requires the delivery of the video over anetwork. In these conventional systems, immersive video must betransferred from a source to a screen. This transfer consumes resourcesbetween the video source and the screen and creates demands on thecomputing resources that host the video storage and the video screen,and on all elements/devices therebetween. Consumption of such resourcesincreases costs and may further limit the quality of the delivered videocontent to the consumer.

The embodiments herein describe and illustrate an immersive video systemthat delivers video, or immersive content, based on saccadic eye motion.Immersive video, or virtual reality, is video that includes some or alla panorama, and when viewed through a head-mounted display or withinsurrounding screens, includes some or all a 360-degree panorama in thehorizontal and/or vertical directions. Immersive video delivered to thevideo display system from the video storage system that is in the samelocation as the video display system, or from a remote location thatrequires the delivery of the video over the network. In either case,immersive video must be transferred from a source to a screen. Thistransfer consumes resources between the video storage system and thevideo display system and creates demands on the computing resources thathost the video storage system and the video display system, and on allelements in between. Systems and methods according to the embodimentsherein are capable of advantageously conserving these resources, therebycreating cost savings for content delivery, as well as better viewerexperiences at a lower cost. The present systems and methods are thusfurther capable of delivering viewer experiences that would otherwise berelatively impossible to deliver on conventional systems.

The embodiments of the immersive video system described herein conservethese resources by tracking the saccadic motion of the eye with the eyetracking video camera. As described further below, an immersive videosystem includes a video display system, an eye tracking video camera, anelectronic network, and a video storage system. The immersive videosystem provides high quality rendering of the immersive video in thevideo display system that the eye will be viewing. The immersive videosystem also provides lower quality rendering of the immersive video inthe area of the video display system that the eye will not be viewing.By providing lower quality rendering in areas that the viewer's eye isnot directly viewing, the present embodiments are better able toconserve resources by sending predominantly high-quality rendering to afield where the viewer's eye will directly view the content.

The biology of the human eye concentrates a very high number of rods andcones in a small area in the center of a focus of each eye, called afoveal area. A diameter of the foveal area is relatively small and canbe limited to a range of approximately two degrees. Human eyes though,have their greatest ability to see detail and color gamut within thisrelatively small area. Additionally, the foveal area of the human eye issurrounded by a blend area having a range of about ten degrees, andgreater diameter, in which the eye has a lower ability to focus andrecognize details. Outer to the blend area is a peripheral vision area,where the human eye has a significantly more limited ability to viewdetail.

Embodiments of the immersive video systems and methods described hereinadvantageously conserve valuable content delivery resources bydelivering higher quality detailed content, i.e., higher resolution, tothe field of a display where the human eye is viewing, and lowerresolution content to the field where the eye is not most directlyviewing (e.g., the blend and peripheral regions). The presentembodiments are thus described below with respect to a display that isdedicated to the use of a single viewer. However, a person of ordinaryskill in the art, after reading and comprehending the presentdisclosure, will understand how the present embodiments may be utilizedwith respect to both eyes of a single viewer or multiple viewers, e.g.,through the use of multiple cameras, or a single camera having theability to track more than one eye at a time. The present systems andmethods are configured to track the movement of one or more human eyesto determine where the eye is looking, and accordingly adjust thedetail/resolution shown on a video display to deliver higher qualitycontent to the fields of the display where the eye is best able to seedetail.

Additionally, the direction of view is not limited to only eye movement,e.g., humans may change their visual field by keeping the head, neck,and body stationary while moving the eyes alone. Humans, however, mayalso change their visual field by a coordinated movement of the head,neck, or body, in addition to just movement of the eyes. This movementof the eyes alone is referred to as a saccade, or saccadic movement.Saccadic movements of the eyes are also referred to as “stereotyped” and“ballistic” movements. Stereotyped saccades refer to the situation wheresaccades generally are known to follow a similar pattern to each other.That is, the velocity of the visual field moving by a saccadic motioncorresponds to an increase in a similarly stereotyped way. Ballisticsaccades refer to the situation where the destination of the movement ispredetermined at the beginning of the movement.

Typical saccade movements made toward a target field of view have a meanduration between 150 ms and 250 ms. The mean duration is long enough forembodiments of the immersive video systems and methods described hereinto detect a start of a saccade movement, predict the destination of thesaccade movement, and react by changing what is being displayed at thetarget field of view prior to the eye's arrival at the target field ofview. As such, embodiments of the immersive video systems and methodsdescribed herein predicts where an eye is going to look based upon thevery beginning of its motion, well before the field of view has reachedthe target field of view.

Embodiments of the immersive video systems and methods described hereinadvantageously adapts the fidelity of displayed content in the targetfield of view based on predicting where the viewer is about to look.Additionally, embodiments of the immersive video systems and methodsdescribed herein adapts the streaming delivery of content to account forthe viewer's current field of view, and the viewer's target field ofview, to reduce the total number of bits that need to be delivered torender the scene for the viewer.

FIG. 1 is a schematic illustration of an exemplary system 100 fordelivering immersive video content for a viewer (not shown). Accordingto the exemplary embodiment, system 100 is an immersive video contentdelivery system for displaying content based, at least in part, onsaccadic eye movement of the viewer. In the exemplary embodiment,immersive video content includes immersive content, video, virtualreality, and immersive video. System 100 includes a client displaydevice or video display system 102, an eye tracking device or eyetracking video camera 104, an electronic network 106, and a videostorage system 108. Video display system 102 includes a video displayprocessor, eye tracking processor, or first processor 110 and a videodisplay 112. Video storage system 108 includes a video storage processoror second processor 114 and a video storage device 116. Electronicnetwork 106 includes one or more of an intra-computer bus, a local areanetwork, an optical network, a hybrid fiber coax network, a wirelessnetwork, or a wide area network such as the Internet. In an exemplaryembodiment, video display 112 includes one or more of a head-mounteddisplay or a plurality of surrounding screens that includes some or alla 360-degree panorama in the horizontal and/or vertical directions.

In an embodiment, video display processor 110 and video display 112 areintegrated in a single device or video display system 102. In analternative embodiment, video display processor 110 is separate fromvideo display 112. In another embodiment, video display processor 110,video display 112, and eye tracking video camera 104 are integrated in asingle device or video display system 102. In an alternative embodiment,video display system 102, eye tracking video camera 104, electronicnetwork 106, and video storage system 108 are integrated into a singledevice or immersive video system 100.

In the embodiment illustrated in FIG. 1, eye tracking video camera 104electronically communicates with video display system 102 by a cameracommunications link 118. In an example of the embodiment, eye trackingvideo camera 104 electronically communicates with video displayprocessor 110 by camera communications link 118. Video display 112electronically communicates with video display processor 110 by a videodisplay communications link 120. Video display system 102 may alsocommunicate with electronic network 106 by a video display processorcommunications link 122. In an exemplary embodiment, video displayprocessor 110 electronically communicates with electronic network 106 byvideo display processor communications link 122. Electronic network 106communicates with video storage system 108 by a network communicationslink 124. More specifically, electronic network 106 electronicallycommunicates with video storage processor 114 by network communicationslink 124. Video storage device 116 electronically communicates withvideo storage processor 114 by a video storage communications link 126.

FIG. 2 illustrates an exemplary content layout 200 for associatingcontent resolution with a viewer's field of view, which may beimplemented with the system depicted in FIG. 1. Content layout 200includes a plurality of viewable display regions 201 about an object 202in layout 200. In the exemplary embodiment, viewable display regions 201include two or more of a foveal region 204, a blend region 206, and aperipheral region 208. In the exemplary embodiment, the viewer (notshown) is focused on object 202, which is centered within foveal region204. For purposes of explanation, foveal region 204 represents theregion on a display (not numbered) of content layout 200 correspondingthe foveal portion of the viewer's eye (containing the relatively highernumber of rods and cones) that centers and focuses its gaze on object202. Foveal region 204 includes a foveal region diameter 210 and issurrounded by blend region 206. Peripheral region 208 surrounds blendregion 206. Foveal region 204, blend region 206, and peripheral region208 are concentrically centered about central point 211. Blend region206 includes a blend region diameter 212.

Content layout 200 also includes a plurality of display resolution zones213. Display resolution zones 213 include two or more of a high-qualityrendering zone 214, a medium quality rendering zone 216, and alow-quality rendering zone 218. That is, high quality rendering zone 214contains relatively higher image resolution than medium qualityrendering zone 216, and medium quality rendering zone 216 containsrelatively higher image resolution than low quality rendering zone 218.As explained further below, an imaging processor (e.g., video displayprocessor 114, FIG. 1) is configured such that display resolution zones213 are concentrically centered substantially about central point 211,such that, in an exemplary embodiment, high quality rendering zone 214substantially overlaps with foveal region 204. In a similar manner,medium quality rendering zone 216 substantially overlaps with blendregion 206, and low-quality rendering zone 218 substantially overlapswith peripheral region 208.

According to the exemplary embodiment, the different diameters of highquality rendering zone 214, medium quality rendering zone 216, andlow-quality rendering zone 218 generally correspond, about central point211, to those of foveal region 204, blend region 206, and peripheralregion 208, respectively. Alternatively, the respective diameters ofdisplay resolution zones may be larger or smaller than those of thecorresponding viewable display regions 201. In the embodiment shown inFIG. 2, high quality rendering zone 214 and medium quality renderingzone 216 are illustrated to be circular regions having a high-qualityrendering region diameter 220 and a medium quality rendering regiondiameter 222, respectively. In an alternative embodiment, high qualityrendering zone 214 and medium quality rendering zone 216 may have anyshape which enables system 100 to operate as described herein,including, without limitation, square and rectangular shapes disposedregularly about central point 211.

In operation, eye tracking video camera 104 is configured to be aimedtoward at least one eye (not shown) of the viewer, and to record videoat a sufficient frame rate to track movement of the eye, for example,240 frames per second (fps). At 240 fps, an image frame is received byvideo display processor 110 approximately every 4 ms. Video displayprocessor 110 is further configured to determine, from the receivedimage frames, the relative position of the field of view within eachimage frame. Thus, if the field of view of the eye begins to shift fromsaccadic motion, video display processor 110 is configured to determinevelocity, acceleration, and direction of travel of the field of view,and further to predict the destination of the field of view. Inexemplary embodiment, the determination of the destination of the fieldof view can be made based on three frames (or 12 ms of samples) from eyetracking video camera 104. Saccadic motion has an average saccadicduration of approximately 200 ms. Thus, on average, video displayprocessor 110 has approximately 188 ms to determine the destination ofthe field of view and to react to that information. The amount of timevideo display processor 110 has to react to the saccadic motion of thefield of view can then be determined as a function of distance of eyemotion travel within the field of view, in degrees.

Embodiments of system 100 and methods described herein may be optimizedlocally or remotely. “Local optimization” refers to the optimization ofthe transfer of data from video display processor 110 to video display112. “Remote optimization” refers the optimization of the transfer ofdata over electronic network 106 between video storage processor 114 andvideo display processor 110. System 100 is configured to furtherconsider a network quality metric to measure the quality of electronicnetwork 106. The network quality metric considers multiple parameters ofelectronic network 106, including latency, congestion, and bit errorrate. Latency, for example, includes the amount of time data takes totravel from one point in electronic network 106 to another point inelectronic network 106. In an embodiment, system 100 further includes apredetermined high-quality area that is the minimum area occupied byhigh quality rendering zone 214 on video display 112.

In further operation, system 100 is configured to engage in localoptimization by changing a resolution, a frame rate, a color gamut, anda dynamic range of the video displayed in content layout 200 of videodisplay 112 to optimize the computing resource load on video displayprocessor 110 and video display 112 by adjusting a bit transferrequirement from video display processor 110 to video display.Accordingly, system 100 advantageously is able to display relativelyhigher resolution video, higher frame rate video, higher color gamutvideo, and/or higher dynamic range video within high quality renderingzone 214 (corresponding to foveal region 204) and immediatelysurrounding area of the field of view, while displaying relatively lowerresolution video, lower frame rate video, lower color gamut video,and/or lower dynamic range video in the other zones of video display 112without significantly reducing the quality of the viewer's visualexperience. That is, high quality rendering zone 214 receives highquality renderings while medium quality rendering zone 216 and/orlow-quality rendering zone 218 receive lower quality renderings,respectively. Nevertheless, the foveal portion of the human eyesubstantially views only the high-quality renderings.

In the exemplary embodiment, video display 112 is configured to displayvideo at a rate of 120 fps in high quality rendering zone 214, whiledisplaying video at a rate of 60 fps to the remaining areas of videodisplay 112, such as medium quality rendering zone 216 and low-qualityrendering zone 218. In the exemplary embodiment, the 120-fps frame rateis selected as a multiple of the 60-fps frame rate to achieve framesynchronization between the respective rendering zones. As the field ofview moves, their eyes view different areas of video display 112. Thesaccadic motion is detected by video display processor 110, and videodisplay processor 110 changes the center (e.g., central point 211) ofhigh quality rendering zone 214 where the high frame rate video isdisplayed. In the exemplary embodiment, a source video file (not shown)is stored at 120 fps. In an embodiment, video display processor 110 isfurther configured to downscale the frame rate to 60 fps in peripheralregion 208 and/or low-quality rendering zone 218.

In exemplary operation, eye tracking video camera 104 tracks theviewer's eye and transmits a video of the viewer's eye to video displayprocessor 110. Video display processor 110 determines a first locationfor the field of view based on at least one frame from the imagesrecorded by eye tracking video camera 104. Video display processor 110then requests a video of stored image frames from video storage system108. Video storage system 108 then transmits the video to video displayprocessor 110. Video display processor 110 then transforms a firstportion of the first received video into a first high-quality video andsecond portion of the video into a first low-quality video. Videodisplay processor 110 then transmits the first high quality video andthe first low quality video to video display 122. Video display 122 thendisplays the first high quality video in high quality rendering zone 214centered on the first location and displays the first low quality videoin medium quality rendering zone 216 and a low-quality rendering zone218, both also centered on the first location.

In an alternative or additional operation, video display processor 110determines a second location for the field of view based on at least oneframe from the image frames recorded by eye tracking video camera 104.Video display processor 110 may then determine a field of viewdestination for the second location based on, for example, at leastthree frames from the image frames recorded by eye tracking video camera104. Video display processor 110 requests a video of stored image framesfrom video storage system 108. Video storage system 108 transmits thevideo of stored image frames to video display processor 110. Videodisplay processor 110 transforms a first portion of the second receivedvideo into a second-high quality video and a second portion into asecond low quality video. Video display processor 110 transmits thesecond-high quality video and the second low quality video to videodisplay 122. Video display 122 displays the second-high quality video inhigh quality rendering zone 214 centered on the second location anddisplays the second low quality video in medium quality rendering zone216 and a low-quality rendering zone 218, both centered on the secondlocation/destination.

In an embodiment, remote optimizations further optimize the bit transferover electronic network 106 between video storage processor 114 andvideo display processor 110. In an exemplary embodiment, electronicnetwork 106 includes one or more of an intra-computers bus, a local areanetwork, and a wide area network such as the Internet.

Referring to FIG. 1, in an embodiment, electronic network 106 is theInternet, and video storage processor 114 is in a data center remotefrom video display processor 110. In operation, video display processor110, with eye tracking video camera 104, determines a location of thefield of view on video display 112 and centers high quality renderingzone 214 (FIG. 2) about the determined location (e.g., central point211). Video display processor 110 is configured to then request videofrom video storage processor 114 having the highest predeterminedresolution, frame rate, color gamut, and/or dynamic range. Video storageprocessor 114 obtains this requested high quality video from videostorage device 116 and transmits the obtained video to video displaysystem 102 for display (e.g., within high quality rendering zone 214,FIG. 2) on video display 112. In a similar manner, lower resolution,frame rate, color gamut, dynamic range is transmitted to video displaysystem 102 for display on video display 112 within lower resolutionrendering zones (e.g., rendering zones 216, 218, FIG. 2).

In an exemplary embodiment, video display 112 is configured to display astandard range of 3840 pixels in the left to right (x) dimension, and2160 pixels in the top to bottom dimension (y). In this example, thehigh-quality rendering zone 214 can thus be configured to occupy 400×400pixels at, for example, a 4K quality resolution, centered on thelocation of the field of view (e.g. central point 211). Further to thisexample, and video display processor 110 is configured to obtain theremainder of the pixel array at a lower resolution, such as 720p, forexample, and map the lower resolution video to the higher resolutionvideo. In operation, when the viewer shifts the field of view to adifferent part of video display 112, video display processor 110 isconfigured to then predict, within an optimal time period, where thefield of view is shifting and re-center the higher resolution renderingzone for that relocated central point 211′ on video display 112. Thesize of the high-quality rendering zone 214 is selected in considerationof measured latency of electronic network 106, such that high qualityvideo is delivered from video storage processor 114 and displayed onvideo display 112 before the field of view arrives at relocated centralpoint 211′.

In exemplary operation, eye tracking video camera 104 tracks theviewer's eye and transmits a video of the viewer's eye to video displayprocessor 110. Video display processor 110 determines a first locationfor the field of view based on at least one frame from the videorecorded by eye tracking video camera 104. Video display processor 110then requests a video of stored image frames from video storage system108. The video of stored image frames a first portion including a firsthigh-quality video and a second portion including a first low-qualityvideo. Video storage system 108 then transmits the first high qualityvideo and the first low quality video to video display processor 110.Video display processor 110 then transmits the first high quality videoand the first low quality video to video display 122. Video display 122then displays the first high quality video in high quality renderingzone 214 centered on the first location for the field of view anddisplays the first low quality video in medium quality rendering zone216 and a low-quality rendering zone 218, both also centered on thefirst location.

In an alternative or additional operation, video display processor 110determines a second location for the field of view based on at least oneframe from the image frames recorded by eye tracking video camera 104.Video display processor 110 may then determine a field of viewdestination for the second location based on, for example, at leastthree frames from the image frames recorded by eye tracking video camera104. Video display processor 110 requests a video of stored image framesfrom video storage system 108. The video of stored image frames includesa third portion including a second-high quality video and a fourthportion including a second low quality video. Video storage system 108then transmits the second-high quality video and the second low qualityvideo to video display processor 110. Video display processor 110 thentransmits the second-high quality video and the second low quality videoto video display 122. Video display 122 then displays the second-highquality video in high quality rendering zone 214 centered on the fieldof view destination and displays the second low quality video in mediumquality rendering zone 216 and a low-quality rendering zone 218, bothcentered on the field of view destination.

In an exemplary operation, the size of high quality rendering zone 214may be optimized based on the latency of electronic network 106. In thisexample, electronic network 106 is determined to have a latency greaterthan 250 ms. That is, data transmitted over electronic network 106 takeslonger than 250 ms to complete a round trip from and to the videodisplay processor. Where, for example, a saccade covers 70 degrees, andhas a duration of 233 ms, the size of high quality rendering zone 214may be optimized such that video display processor 110 has sufficienttime to detect the motion of the saccade and transmit a request to videostorage processor 114 to change the focus location and/or resolutionzone size. Alternatively, in a case where electronic network 106 has alatency less than 125 ms, optimization of the size of high qualityrendering zone 214 may be more easily optimized with respect to, forexample, a saccade covering 30 degrees with a duration of approximately100 ms. The present embodiments are thus further advantageously able tomeasure the network latency and utilize the measured latency as aparameter to set the resolution quality for the several rendering zones.

Thus, in the example above, when electronic network 106 is measured toprovide a generally consistent latency of approximately 125 ms or less,high quality rendering zone 214 is sized such that system 100 may set orshift the location of high quality rendering zone 214 before thedetected field of view can move outside of high quality rendering zone214. In one embodiment, the size of high quality rendering zone 214 isset, for example, to cover up to approximately 30 degrees in anydirection from central point 211. That is, high quality rendering zone214 may include a 30-degree radius circle on video display 112 centeredabout central point 211, with all other areas of video display 112designated for medium quality rendering zone 216 and/or low-qualityrendering zone 218. According to the exemplary embodiment, as themeasured latency of electronic network 106 increases, video displayprocessor 100 correspondingly reduces the size of high quality renderingzone 214. Accordingly, system 100 is further advantageously configurableto dynamically react to the measured latency of electronic network 106by increasing or decreasing the size of high quality rendering zone 214.

In general, human eye motion is faster than human head, neck, or bodymotions. According to an alternative embodiment, the systems and methodsdescribed herein are implemented for video displays 112 including aplurality of surrounding screens (as opposed to a wearable viewscreen)that includes up to a 360-degree panorama in the horizontal and/orvertical directions. Referring to the previous example, in thisalternative embodiment, video display processor 110 may set the size ofhigh quality rendering zone 214 to be smaller than 30 degrees, since thehuman head viewing a 360-degree panorama (or smaller) will not move asquickly as the human eye, and thus system 100 will not generally have torespond as quickly to relocate central point 211 as it would have tobase on saccadic eye motion alone.

Referring again to FIG. 1, a process 300 of optimizing video deliverybased upon saccadic eye motion is also illustrated. Process 300 includesa detecting step 302, using eye tracking video camera 104, a field ofview of at least one eye of the viewer, and thereby transmitting videodisplay coordinates from the detected field of view to a video displayprocessor 110. An identifying step 304 identifies a region on videodisplay 112 corresponding to the transmitted video display coordinatesand then further requesting, by video display processor 110, theimmersive content from video storage processor 114 at a first resolution(not separately numbered) for a first portion of the immersive contentand a second resolution (not separately numbered) for a second portionof the immersive content. In this exemplary process, the firstresolution is higher than the second resolution. For example, the firstresolution may represent a high-quality image rendering, such as 4 k,and the second resolution may represent a relatively lower quality imagerendering, as discussed above. Additionally, a third resolution, lowerthan the second resolution, may be implemented, for example, withrespect to the outer peripheral region 208 or low-quality rendering zone218.

Process 300 then proceeds to a receiving step 306, where video displayprocessor 110 receives the first portion of the immersive content at thefirst resolution and the second portion of the immersive content at thesecond resolution, and then centers the corresponding display of thefirst and second portions of the immersive content about central point211 of the region identified in step 304. In step 308, process 300displays the centered corresponding first portion of the immersivecontent on video display 112 within a zone of the video displayincluding central point 211 and displays the centered correspondingsecond portion of the immersive content on video display 112 outside ofthe zone occupied by the first portion of the immersive content.According to these advantageous processing embodiments, the data volumeof the combined first and second video portions is significantly lowerthan a data volume if both the first and second portions were renderedat the same higher resolution. By utilizing measured saccadic eye motiondata, alone or together with other parameters such as network latency,the present embodiments can significantly reduce the quantity of datatransmitted, and the time to render the data on a display, in virtualreality applications without significantly reducing the quality of thecontent experienced by the viewer.

One non-limiting aspect of the present invention contemplates a visualcoding system that provisions signaling and adjustment of intensity andresolution of light and/or color visual information in a rendered viewbased on the aspects of focal and peripheral vision relative to theposition of eye. The visual coding system may take into considerationperipheral vision being more light sensitive and accordingly control bitallocation to favor luminance vs. color information. And whereas thefovea receives images in a direct line of vision that has the maximumresolution of cones which are less sensitive to light, and moresensitive to color, correspond the control bit allocation to favor colorinformation within the fovea. The visual coding system may scale imagesunequally or inconsistent across an entirety of a corresponding frame soas to vary resolution, luminance and/or chrominance according to relatedsensitivities of the human visual system (HVS). The in-frame or perframe variance in how images are coded throughout its metes and boundsmay be beneficial in leveraging or expending bits according tocapabilities of the HVS, which may include limiting bits allocated forcolor to portions of the individual image frames that the viewer's HVScan appreciate, e.g., limiting or reducing the number of bits allocatedfor color/chrominance within a peripheral vision of the user due to arelative inability of the viewer to appreciate color/chrominance withinthe peripheral vision. Such varying of coding across a frame may beaccomplished by encoding segments of the frame in the peripheral (morelight sensitive, less resolution) and segments of the frame in the fovea(more color sensitive, more resolution) differently. Techniques such as“tiling” within a frame also accomplish this varying encoding.

Our ability to sense color diminishes rapidly as the field of viewapproaches our periphery vision, such that color is essentiallyirrelevant (there are no cone photoreceptors) in the periphery visionand it need not be allocated any bits. When one looks directly at anobject, the image is projected onto the fovea whereat the fovea hasmaximal visual acuity (high resolution) and a high density of cones(specialized photoreceptors to sense colors but relatively insensitiveto light). If one looks at the same object using peripheral vision, theimage is formed more eccentrically on the retina since the cones aresparse in the peripheral vision but high densities of rods are present.Rods do not detect color, but are much more sensitive to light thancones, and many rods converge onto a single ganglion cell such that rodresponses can be summated to further increase light sensitivity in theperiphery while degrading spatial resolution. One non-limiting aspect ofthe present invention contemplates allocating more bits to thechrominance information in the visual signal that is targeted for thefovea than the number of bits allocated for the visual signal targetedfor the peripheral portion of the retina. That is, chrominanceinformation is less relevant in our peripheral vision, and thereforebits that are allocated to the carriage of chrominance information forour peripheral vision are wasted bits. This capability may provide amore realistic visual experience that can allocate: higher spatialresolution in the signal targeting the fovea (foveated rendering);higher chrominance resolution in the signal targeting the fovea; andlower or almost no bits (i.e. low color and/or resolution) forchrominance in the signal targeting the outer portions of the retina.

The in-frame coding variances (e.g., lower chrominance in the peripheralvision) may operate in coordination with foveated rendering capabilitiesfor allocating bits to the area in focus with higher spatial resolutionand lower spatial resolution for the periphery. This capability may beused to facilitate reducing or lowering chrominance within theperipheral vision relative to the non-peripheral vision while alsoreducing image resolution within the peripheral vision relative to thenon-peripheral vision. Coded video stream and coding system that tilesor otherwise segment portions of each frame may be utilized tofacilitate the contemplated in-frame coding variances, e.g., eachtile/segment within the metes and bounds of a particular image frame maybe independently coded in layers for resolution, luminance andchrominance. Based on the position of the viewer's gaze (which issignaled by the display), an encoder/renderer can choose to allocatemore resolution and color information to the tiles/segments that aremapped to the center of the viewer's gaze, and less color, i.e. morelight information, for the tiles/objects mapped to the viewer'speripheral vision. A more realistic visual experience based on eyetracking or saccadic movement of the viewer may be employed tofacilitate adjusting where the viewer's peripheral vision lies withrespect to a display or other interface associated with rendering theimage frames.

FIG. 3 illustrates a video display system 10 in accordance with onenon-limiting aspect of the present invention. The video display system10 is exemplarily illustrated with respect to being an immersive, AR/VRor other video device 12 worn by a viewer. The system 10 may include aplurality of eye tracking cameras 14 or sensors to facilitate monitoringsaccadic movement of the viewer and their attendant field of view, whichis shown with respect to a fovea region 16, a blend region 18 and aperipheral region 20. The illustrated regions 16, 18, 20 within theviewer's field-of-view or gaze may be spatially related to first andsecond displays 24, 26 within which corresponding video images may bedisplayed. Positional coordinates of x, y may be used to defineboundaries/areas for the viewing regions 16, 18, 20 as well as formatching segments on the displays 24, 26. The x, y coordinates for theviewing regions 16, 18, 20 may be used to map to related x, ycoordinated portions of the displays, which in turn may be mapped tocorresponding segments, tiles or other portions of an image frame 28displayed therein. The viewer's field-of-view and the displays 24, 26may be relatedly divided into segments or tiles for purposes ofgenerating a relationship sufficient to relate the regions 16, 18, 20 toparticular segments or tiles of the individual image frame 28, which maybe each comprised of one or more pixels. The metes and bounds associatedwith each of the field of view, displays 24, 26 and the image frame 28may be mapped relative to the eye tracking cameras 14 to facilitatecoding individual portions of each image frame 28 relative to saccadicmovement and sensitivities of the HVS.

The video display system 10 may include a calibration process whereby aviewer is instructed to undertake certain eye movements while wearingthe device 12 for purposes of assessing individual nuances in theviewer's HVS, i.e., calibrate where the foveal, blend and peripheralregions 16, 18, 20 begin and end for individual viewers. The calibrationprocess may also accommodate age of the viewer or other sensitivitiesthat may affect how well the associated viewer can appreciate luminanceand/or chrominance within certain portions of their field-of-view. Thebitstream being coded to render the visual information via the displays24, 26 may be flexible such that layers of bits can be progressivelyadded to the rendered data (color or light information, additionalspatial resolution) based on the center of focus/peripheral vision ofthe viewer. The attendant coding may optionally be performed inaccordance with the capabilities and operations described in U.S. Pat.No. 9,100,631, entitled Dynamic Picture Quality Control, the disclosureof which is hereby incorporated by reference in its entirety herein,whereby various control parameters may be utilized to facilitate codingan uncompressed image into a compressed image at a rate sufficient toenable the attendant images to be sequentially displayed as video. Thecontemplated coding may facilitate in-frame differences being demarcatedaccording to the illustrated boundaries between the foveal, blend andperipheral regions 16, 18, 20 or according to less demarcated or gradualchanges, e.g., gradients could differentiate portions of thefield-of-view rather than the illustrated boundaries.

FIG. 4 illustrates a flowchart 40 for an optimized method for visualcoding for sensitivities to light, color and spatial resolution in theHVS in accordance with one non-limiting aspect of the present invention.The operations, processes and other manipulations associated with theillustrated flowchart may be facilitated entirely or partly with aprocessor of a renderer/encoder, or other element associated therewith,executing according to a corresponding plurality of non-transitoryinstructions stored on a non-transitory computer-readable medium. Themethod is predominately described for illustrative and exemplarypurposes with respect to the HVS including sensitivities to light, colorand spatial resolution throughout its field-of-view. The method isdescribed with respect to making in-frame adjustments for luminanceand/or chrominance, which may occur with or without correspondingin-frame resolution adjustments. The disparate resolutions within aparticular image frame may be beneficial for the reasons describedabove, however, one non-limiting aspect of the present inventioncontemplates disparate luminance and/or chrominance variances within aparticular image frame achieving benefits independent of those notedabove such that the contemplated chrominance/luminance variances may beimplemented in addition to or without of resolution variances.

Block 42 relates to determining a field of view for a particular user,such as with a calibration process or other operation executed while theuser is wearing the device. The field-of-view may be broken down intoany number of regions, such as the above-described foveal, blend andperipheral regions, or according to other mapping techniques sufficientfor identifying segments, tiles or particular areas within thefield-of-view subjected to the noted sensitivities of the HVS. Thefield-of-view determination may take into consideration a size/shape ofthe display 24, 26 and the image frames 28 to create a relationshiptherebetween sufficient for correlations with the HVS regions 16, 18, 20of sensitivity. FIGS. 5-6 illustrate a relationship between the HVSregions 16, 18, 20 of sensitivity and the display 24, 26 and the imageframes 28 being sufficient for instructing an encoder/renderer todisparately code each image frame 28 such that different segments 46,48, 50 of the same image frame are coded differently from the othersegments 46, 48, 50 depending on whether the attendant segment fallswithin one of the HVS regions 16, 18, 20, e.g., depending on whether thesegment 46, 48, 50 is aligned or coincidental with the foveal, blend orperipheral region 16, 18, 20, or more generically whether the segment46, 48, 50 is aligned or coincidental with a peripheral region/vision 20or a non-peripheral region/vision 16, 18.

Block 54 relates to coding image frames 28 according to the determinedfield-of-view. The coding may correspond with individually coding imageframes 28 to be shown in sequence as a video such that segments, tilesor other divisions 46, 48, 50 within each image frame 28 are disparatelycoded with some portions of the same image frame being coded differentlythan other portions. One non-limiting aspect of the present inventioncontemplates segmenting or otherwise partitioning the individual imageframes 28 according to virtually any level of granularity, such as bydividing the image frames into a number of individual segments 46, 48,50. Individual segments 46, 48, 50 may then be grouped or associatedwith one of the HVS regions 16, 18, 20, e.g., multiple segments may beassociated with each of the regions 16, 18, 20 or alternatively, thesegments 46, 48, 50 as illustrated may be aligned with the HVS regions16, 18, 20, i.e., such that each segment corresponds with one of the HVSregions (foveal, blend, peripheral, non-peripheral, etc.). The codingmay generally correspond with generating a bitstream or other sufficientrepresentation of the individual images to facilitate subsequentrendering according to the processes contemplated herein. The coding maybe performed at a rate sufficient to facilitate making adjustments on aframe-to-frame basis such that the disparate coding in one image framemay be varied or changed in a next image frame.

Returning to the FIGS. 5-6, the image frames 28 being coded for the leftand right eye may include three segments 46, 48, 50 of disparate codingto coincide with the foveal, blend and peripheral regions 16, 18, 20.The coding made occur at a sufficient granularity to enable theillustrated demarcation along corresponding boundaries defined in the x,y coordinate system or through other, more gradual scaling away from acenter or central point of the viewer's gaze. One non-limiting aspect ofthe present invention contemplates utilizing coding tools to define acolor gamut for the image frames 28, such as according to a Y, Cr, Cbtype of color gambit whereby bit values are assigned for each of Y, Cr,Cb to define the luminance and chrominance associated therewith. Thenumber of bits assigned to each of the values may be referred to as acolor depth, i.e., number of bits needed to uniquely represent all ofthe possible colors that are in the media, and represented as bits perpixel (BPP), bits per component, bits per channel, bits per color, etc.The number of bits assigned to each of the Y, Cr, Cb values represent alevel of coding according to customary color gambit scaling used torepresent a quantization, color or shading variances/possibilities ofcolor, e.g., a one 1-bit color, 2-bit color, 3-bit color . . . n-bitcolor. The corresponding color depth or n-bit color may bedifferentiated according to known color groupings, such as monochrome(1-bit color), standard dynamic range (SDR) (8-bit color), high dynamicrange (HDR)(10-bit color), high color (15/16-bit color), true color(24-bit color), and Wide Color Gamut (WCG), etc.

An entirety of the display 24, 26 may be occupied by a singular imageframe 28 having a coded resolution defining a resolution of the display(assuming the display includes capabilities sufficient to match thecoded resolution). The illustrated frame 28 may be encoded as: a segment50 associated with the peripheral region including a Y value of 10-bit,a Cr value of 8-bit and a Cb value of 8-bit (10, 8, 8); a segment 48associated with a non-peripheral region including a Y value of 10-bit, aCr value of 10-bit and a Cb value of 10-bit (10, 10, 10); and optionallyif further granularity is desired an additional segment 46 associatedwith an additional non-peripheral region including a Y value of 10-bit,a Cr value of 12-bit and a Cb value of 12-bit (10, 12, 12). These BPPvalues are merely exemplary of the capabilities of the present inventionto disparately code at virtually any granularity throughout the metesand bounds of a singular image frame 20. The presented Y, Cr, Cb valuesindicate a consistent luminance or matching luminance throughout theimage frame 28, i.e., the peripheral and non-peripheral regionsincluding the same 10-bit value for Y for exemplary purposes ofdescribing one encoding methodology where luminance is consistentthroughout. Each of the Y, Cr, Cb may be varied to produce other codingresults, including varying the Y value between the peripheral andnon-peripheral regions so as to adjust the luminance associatedtherewith, e.g., having a greater Y value in the peripheral region wheresusceptibility to luminance is better.

A lesser amount of data would be required to transmit a particular imageframe 28 having lower BPP values than the same image frame 20 wouldrequire if coded at greater BPP values such that the capability tomaintain higher BPP values in certain portions of the image frame 28where luminance/chrominance may be better perceived and lower BPP valuesat other portions where luminance/comments may be less perceptible canmaximize data usage for conveying images in a manner most useful givingsensitivities of the HVS. This capability, in comparison to codingsystems that uniformly or consistently code image frames with the sameluminance/chrominance throughout, may be beneficial in decreasing theamount of data needed to transmit the same images due to some bit valuesbeing reduced and/or to utilize the data saved by reducing some of thebit values to improve resolution and/or add more luminance/chrominanceat desirable portions of the image to make the attendant video morerealistic, i.e., to facilitate transmission of the video with the sameamount of data consumption but with the data use being maximizedrelative to sensitivities of the HVS so as to provide a more realisticrendering at least in the sense of the luminance and chrominance beingincreased/decreased throughout an image frame to account forcapabilities of the HVS to perceive luminance/chrominance within aparticular field of view.

Returning to FIG. 4, Block 56 relates to monitoring the saccadicmovement of the viewer as images are being successfully rendered as partof a video presentation to determine whether a viewer's gaze hasshifted. FIG. 5 illustrates the user's gaze being essentiallystraightforward or centered within the displays 24, 26 of the devicesuch that a center of the foveal aligns with a center of thecorresponding display. The viewer's gaze may shift over time with thefoveal moving in virtually any direction up, down, left, right or somecombination thereof from the illustrated central location such that thefoveal is centered or looking towards a different section or quadrant ofthe display. This eye movement may result in a portion of the peripheralregion moving beyond a boundary of the corresponding display or produceother adjustments in the size and shape of the HVS regions. FIG. 6illustrates saccadic movement where the peripheral region on a left halfof the display increases with rightward movement of the fovea. One ofthe preceding processes may be returned to in response to the saccadicmovement to facilitate making appropriate adjustments in the coding ofimage frames to be displayed thereafter, i.e., changing the specialrelationship of the HVS portions within successive frames.

As supported above, one non-limiting aspect of the present inventioncontemplates inconsistently or disparately performing in-frame orper-frame coding so as to adapt individual portions of individual framesaccording to sensitivities of the HVS. This may be particularlybeneficial in maximizing bit usage and data consumption according tosensitivities of the HVS, such as by favoring chrominance over luminancewithin the peripheral region of a viewer's gaze. Various aspects of thedescription herein make reference to visual media, video and other formsof image rendering for exemplary non-limiting purposes as the presentinvention fully contemplates its use and application in facilitatingcoding of any media according to sensitivities of the HVS, particularlyaccording to the above-described sensitivities associated withchrominance and luminance recognition limitations of the human eye.

Exemplary embodiments of immersive video systems and methods aredescribed above in detail. The systems and methods of this disclosurethough, are not limited to only the specific embodiments describedherein, but rather, the components and/or steps of their implementationmay be utilized independently and separately from other componentsand/or steps described herein.

Although specific features of various embodiments of the disclosure maybe shown in some drawings and not in others, this convention is forconvenience purposes and ease of description only. In accordance withthe principles of the disclosure, a feature shown in a drawing may bereferenced and/or claimed in combination with features of the otherdrawings.

Some embodiments involve the use of one or more electronic or computingdevices. Such devices typically include a processor or controller, suchas a general purpose central processing unit (CPU), a graphicsprocessing unit (GPU), a microcontroller, a reduced instruction setcomputer (RISC) processor, an application specific integrated circuit(ASIC), a programmable logic circuit (PLC), a field programmable gatearray (FPGA), a digital signal processing (DSP) device, and/or any othercircuit or processor capable of executing the functions describedherein. The processes described herein may be encoded as executableinstructions embodied in a computer readable medium, including, withoutlimitation, a storage device and/or a memory device. Such instructions,when executed by a processor, cause the processor to perform at least aportion of the methods described herein. The above examples areexemplary only, and thus are not intended to limit in any way thedefinition and/or meaning of the term “processor.”

This written description uses examples to disclose the embodiments,including the best mode, and to enable any person skilled in the art topractice the embodiments, including making and using any devices orsystems and performing any incorporated methods. The patentable scope ofthe disclosure is defined by the claims, and may include other examplesthat occur to those skilled in the art. Such other examples are intendedto be within the scope of the claims if they have structural elementsthat do not differ from the literal language of the claims, or if theyinclude equivalent structural elements with insubstantial differencesfrom the literal language of the claims.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms of the invention. Rather,the words used in the specification are words of description rather thanlimitation, and it is understood that various changes may be madewithout departing from the spirit and scope of the invention.Additionally, the features of various implementing embodiments may becombined to form further embodiments of the invention.

What is claimed is:
 1. A method for visual coding for sensitivities tolight, color and spatial resolution in a human visual system (HVS), themethod comprising: determining a field of view for the HVS; determininga visual media to be displayed within the field of view, the visualmedia including a plurality of image frames; and coding the plurality ofimage frames to account for sensitivities of the HVS varying across thefield of view.
 2. The method of claim 1 further comprising: dividing thefield of view into at least a first portion and a second portion, thefirst and second portions representing differing sensitivities of theHVS; and coding first and second segments within each of the pluralityof frames according to the sensitivities of the HVS, the first andsecond segments respectively coinciding with the first and secondportions.
 3. The method of claim 2 further comprising coding the firstand second segments such that one includes more chrominance than theother.
 4. The method of claim 2 further comprising coding the first andsecond segments such that one includes more luminance than the other. 5.The method of claim 2 further comprising coding the first segments toinclude a greater color depth than the second segments, the color depthbeing measured as bits per pixel (BPP).
 6. The method of claim 2 furthercomprising coding the first and second segments according to a Y, Cr, Cbtype of color gamut, including coding the first segments to includevalues for at least one of the Y, Cr, Cb different than the secondsegments.
 7. The method of claim 2 further comprising coding the firstsegments to include more color than the second segments.
 8. The methodof claim 2 further comprising coding the first segments to include aresolution matching the second segments.
 9. The method of claim 2further comprising coding the first segments to include less light thanthe second segments.
 10. The method of claim 1 further comprising:segmenting each of the plurality of image frames according to the fieldof view and the sensitivities of the HVS such that each of the pluralityof image frames includes at least a first segment associated with afirst sensitivity of the HVS and a second segment associated with asecond sensitivity of the HVS; and coding the plurality of image framessuch that the first segment of each is coded differently than the secondsegment to account for the first and second sensitivities of the HVS.11. The method of claim 10 further comprising altering according tosaccadic movement of the viewer a first spatial relationship between thefirst and second segments within a first frame of the plurality of imageframes relative to a second spatial relationship between the first andsecond segments within a second frame of the plurality of image frames.12. The method of claim 11 further comprising coding the second segmentwithin in each of the plurality of image frames to include lesschrominance than the first segment.
 13. A method for coding image framescomprising: determining a field of view for a viewer to include aperipheral region and a non-peripheral region; and coding the imageframes to include more chrominance than luminance within thenon-peripheral region than the peripheral region.
 14. The method ofclaim 13 further comprising defining the chrominance according to bitsper pixel (BPP) such that the peripheral region includes a lower BPPthan the non-peripheral region.
 15. The method of claim 13 furthercomprising defining the chrominance according to according to a Y, Cr,Cb type of color gamut such that the peripheral region includes valuesfor at least one of Cr and Cb less than values for at least one of Crand Cb of the non-peripheral region.
 16. The method of claim 13 furthercomprising defining the chrominance according to High dynamic range(HDR) such that the peripheral region includes a non-HDR and thenon-peripheral region includes an HDR.
 17. The method of claim 13further comprising coding a resolution of the peripheral region to matcha resolution of the non-peripheral region.
 18. The method of claim 13further comprising coding a resolution of the peripheral region to beless than a resolution of the non-peripheral region.
 19. The method ofclaim 13 further comprising: monitoring saccadic movement of the viewer;identifying the peripheral region within a first image frame of theimage frames to correspond with a first portion of a display; andidentifying the peripheral region within a second image frame of theimage frames occurring after the first image frame to correspond with asecond portion of the display, the second portion being spatiallyseparated from the first portion in proportion to the saccadic movement.20. A non-transitory computer-readable medium having a plurality ofnon-transitory instructions executable with a processor of an encoder tofacilitate optimizing video coding for sensitivities to light, color andspatial resolution of a human visual system (HVS) within a peripheralvision of a viewer, the non-transitory instructions being sufficientfor: individually coding image frames comprising the video according tosaccadic movement of a viewer such that each portion of each image framebeing displayed within the peripheral vision of the viewer includes lesschrominance than other portions of the corresponding image frame to bedisplayed outside of the peripheral vision.